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ABSTRACT 

Space Station Crew Health Care System 
procedures require the use of an on-board 
microscope whose slide images will be 
transmitted for analysis by ground-based 
microbiologists. Focusing of microscope slides 
is low on the list of crew priorities, so NASA is 
investigating the option of telerobotic focusing 
controlled by the microbiologist on the ground, 
using continuous video feedback. However, 
even at Space Station distances, the 
transmission time lag may disrupt the focusing 
process, severely limiting the number of slides 
that can be analyzed within a given bandwidth 
allocation. Substantial time could be saved if 
on-board automation could pre-focus each 
slide before transmission. The authors 
demonstrate the feasibility of on-board 
automatic focusing using a fuzzy logic rule- 
based system to bring the slide image into 
focus. The original prototype system was 
produced in under two months and at low cost. 

Slide images are captured by a video camera, 
then digitized by gray-scale value. A software 
function calculates an index of "sharpness" 
based on gray-scale contrasts. The fuzzy logic 
rule-based system uses feedback to set the 
microscope's focusing control in an attempt to 
maximize sharpness. 

The system as currently implemented performs 
satisfactorily in focusing a variety of slide types 
at magnification levels ranging from lOx to 
lOOOx. Although feasibility has been 

demonstrated, the system’s performance and 
usability could be improved substantially in four 
ways: by upgrading the quality and resolution 
of the video imaging system (including the use 
of full color); by empirically defining and 
calibrating the index of image sharpness; by 
letting the overall focusing strategy vary 
depending on user-specified parameters; and 
by fine-tuning the fuzzy rules, set definitions, 
and procedures used. 


INTRODUCTION 

The Space Station Crew Health Care System 
periodically collects microscope slides that 
must be analyzed by microbiologists on the 
ground within hours of collection. It would be 
desirable to minimize the time required for the 
crew to mount and focus microscope slides. 
However, simple telerobotic control by the 
microbiologist under visual feedback would 
entail several problems: first, a transmission 
delay of about two seconds in each direction 
will disrupt the focusing process, producing 
both slower and less satisfactory results than 
hands-on control would; second, because of 
the additional time required, the focusing 
process itself would tie up significant amounts 
of video downlink bandwidth, reducing the time 
available for the actual analysis; finally, the 
requirement to schedule slide mounting, 
focusing, image transmission, and analysis at 
the same time places tight constraints on all 
resources used, and may be highly inefficient 
and inflexible. 

The system would be far more robust and 
efficient if the crew could collect the slide 
specimens and place the slides in a cassette or 
"jukebox", with no further crew attention 
required. An on-board robot could then mount 
each slide on the microscope, bring the slide 
into focus, and transmit one or more digitized 
still images to a computer on the ground. The 
microbiologists could then view and analyze 
the slides at their convenience. The resulting 
digitized images would require far less 
bandwidth to transmit than a continuous video 
image, yet each digital image could have better 
resolution and less noise than standard video. 

Of course, in practice the microbiologist might 
still wish to be part of the focusing loop, either 
to concentrate on particular areas within a slide 
or to fine-tune focusing when the system's 
focus does not seem sharp enough. This could 
be accommodated by exception (recalling a 
slide that is still available in the "jukebox"), or 
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by allowing the microbiologist to preview each 
slide (in real time) and transmit a command that 
would modify or override the system's 
automatic focusing. Alternately (at greater cost 
but probably still less than full-video 
transmission), the system could transmit not 
one "focused" image but a series of images 
representing a range of focus settings. 

The major technical challenge in this approach 
is the automatic focusing itself; mature 
technologies exist to handle the slide 
mounting, and to digitize, transmit, store, and 
display still slide images. This paper describes 
a demonstration prototype system that the 
authors developed at McDonnell Douglas 
Space Systems Company, with significant 
contributions by Apt Instruments, Incorporated 
(now Aptronix). The system demonstrates the 
feasibility of automatic focusing using fuzzy 
rule-based control and the possibility of 
producing similar control systems rapidly and 
inexpensively. 

SYSTEM DEFINITION 


Functions 

Functionally, the system operates as a closed- 
loop discrete-time feedback system whose goal 
is to maximize the sharpness of a slide image 
by controlling the vertical position of the 
microscope stage {see Figure 1). 



FIGURE 1 

Slide Images are captured by video ana 
digitized. The system calculates a numerical 
scalar value representing the "sharpness" of 
any image from its digital record. This 
sharpness index and the difference between its 
current and previous values form the inputs to a 
system of fuzzy "if-then" rules that determines 
whether the maximum sharpness has been 
reached. If if has, the system stops and displays 
the resulting image. If the system has not 
reached its stopping point, the rule-based 


system specifies the amount and direction to 
adjust the microscope’s focusing control. The 
result produces a new image, which starts the 
cycle again. 

Hardware 

Figure 2 shows the hardware used in the 
demonstration prototype system. This system 
was assembled to the authors' specifications 
by Apt Instruments, Inc., under contract to 
McDonnell Douglas Space Systems Company. 
An Olympus microscope with lOx, 100x, and 
lOOOx magnification levels produces slide 
images that can be viewed directly, but are also 
transmitted to a Javelin black-and-white solid- 
state video camera. The video image is 
captured by a frame digitizer board on the 
80386 personal computer. The digitized image 
can be displayed on a video monitor. 

The 80386 personal computer performs the 
computations necessary to interact with the 
user, to compute the sharpness index, to run 
the fuzzy logic inference engine, and to specify 
desired focusing commands. These 
commands, in digital form, are transmitted to a 
motor controller, which translates them into 
pulses that drive a stepper motor. This motor, 
permanently mounted to the microscope's 
focusing mechanism, adjusts the vertical 
position of the microscope’s stage. 

Software 

Although much of the system's software covers 
routine functions such as system setup, input- 
output, etc., four components are worth 
describing in detail. 


USER MENU 

The user’s interaction with the microscope 
system operates from a main menu of options. 
These are selected by pressing the function 
keys on the keyboard and are presented in 
logical sequence. 

SET/RESET - This must be performed at 
system startup and whenever a new objective 
lens is selected. The user may also RESET the 
system at any time. The system promptsjthe 
user to select a level of magnification’ (The 
current system requires the user to rotate the 
objective lenses manually; if the appropriate 
hardware were available it would take only 
minor software changes to control the lenses 
robotically.) Once a level of magnification has 
been selected, the user is prompted to set the 
absolute limits on the height of the 
microscope‘s slide stage. This prevents 
physical damage to the microscope or to the 
slide. (Again, a more sophisticated robotic 
system could perform this task automatically.) 
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SYSTEM HARDWARE CONFIGURATION 

I. VIDEO CAMERA JE2J62 2. MICROSCOPE BH-2 J. MOTOR MJ7-3I 4 . MACHINE CONTROLLER 
5. IBM PERSONAL COMPUTER AT 6. FRAME GRABBER BOARD DT2I55 7. MONITOR 

FIGURE 2 


FOCUS - FOCUS initiates the automatic 
focusing procedure, adjusting the height of the 
microscope stage until the sharpest image has 
been reached. When selected right after 
SET/RESET, FOCUS uses the entire digitized 
slide image as input to the computation of the 
sharpness index. When selected after AREA, it 
uses only the designated area(s). 

MANUAL - In MANUAL operation mode lets the 
user control the microscope focus directly, 
using the computer keyboard. The "up" and 
"down" arrow keys move the stage up and 
down in fine gradations, while the "page up" 
and "page down" keys move in larger steps. 
After each keystroke, the stage is moved and 
the new image displayed. 

AREA - This option permits the user to 
customize the focusing process by specifying 
up to three areas of the slide image. When one 
or more areas have been specified, portions of 
the picture outside such areas are disregarded 
when computing the sharpness index. This 
allows the user to exclude irrelevant objects, 
dust or scratches on the slide’s cover slip, etc., 
which might otherwise interfere with the 
focusing calculations; it is also valuable at high 
magnifications, when the slide image may have 


objects that overlap at different depths or lie at 
an angle to the focal plane. Areas are selected 
by using the computer’s mouse to draw an 
outline which is superimposed on the 
displayed digital image. Users also have the 
option to rate the "importance" (impact on the 
sharpness index) of the areas selected, 
choosing Very high’, 'high', or ’medium’. Once 
AREA has been completed, the next step is to 
select FOCUS again; because the sharpness 
index is computed differently, the FOCUS 
routine will in general select a different stage 
position as the best focus. 

COLLECT DATA - The user is prompted to 
specify a (rectangular) area of the displayed 
image, using the computer’s mouse. The 
selected area of the digital image is saved as a 
data file on the computer. This data could be 
used to provide a digital image that can be 
annotated or enhanced off-line, to document 
system activity or performance, or to provide 
input to image analysis programs. 

RETRIEVE/DISPLAY IMAGE - The user can 
view a previously saved image file on the video 
display. 

MODE - The user may select ’continuous’ or 
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'single step'. ’Continuous 1 mode, which 
focuses the image without further action on the 
user's part, is the system’s normal modeof 
operation. In 'single step' mode, the system 
stops after each focusing adjustment, giving the 
user time to view the image, to return to 
MANUAL mode, or possibly to save the image 
using the COLLECT DATA option. 


COMPUTATION OF THE SHARPNESS INDEX 

At the foundation of the focusing system is the 
idea that an image (represented as a 
rectangular array of digital gray-scale pixels) 
can be evaluated by computing a scalar-valued 
"sharpness index." This concept is appealing 
because once a suitable sharpness function 
has been defined, the problem of focusing can 
be reduced to finding the highest value for a 
function of a single variable (the height of the 
microscope stage) whose value is restricted to 
a known interval. Although there are a number 
of subtleties, technical difficulties, and empirical 
issues still to be addressed, the current 
demonstration prototype uses a fairly simple 
general-purpose sharpness function with 
generally successful results. (Some of these 
issues, with possible alternate approaches, are 
described later in this paper.) 

The program calculates sharpness in the 
following manner: 

(1) Reduce computation by sampling only a 
small number of lines from the image. 
Currently, the program samples 7 bands 
of three adjacent lines each. 

(2) Treat each of the bands in the sample as 
a set of overlapping 3x3-pixel squares. 
For each 3x3 square, represent the gray- 
scale values as a through /, in the 
following configuration: 



(3) Compute horizontal contrast as 

x /7 = (c+2f+i) - (a+2d+g), and vertical 
contrast as = (a+2b+c) - (g+2h+i). 
Total squared contrast for each 3x3 
square is computed as 

*t ot 2 = *h 2 + x v 2 - 

(4) Sum the total squared contrasts for all 3x3 
squares in the sampled bands to arrive at 
the raw sharpness index. 

Essentially, this formula is the sum of squared 
local gray-scale contrasts, computed on a 
sample of the total image. For scaling 
purposes, the program also computes a 


normalized sharpness index, assigning a value 
of 100 to the best-rated image that is scanned 
during the SET/RESET routine as the user 
specifies the highest and lowest possible stage 
positions. The rationale for the current method 
is that as a well-focused image is blurred, 
sharp local contrasts will turn into smoother 
gradients, thereby lowering the overall sum of 
squared local contrasts. 

IMPLEMENTATION OF FOCUSING STRATEGY 

The focusing strategy employed by the system 
is based on a set of fuzzy rules. At each step in 
the focusing process, the current value of the 
normalized sharpness index and the difference 
between the current and the previous values 
are inputs to a fuzzy inference process 
described below. The output of that process is 
a suggested magnitude and direction for 
adjusting the microscope. For certain values 
(corresponding to well-focused images that do 
not show significant improvement with further 
adjustment), the rule-based system will suggest 
a change of zero; at that point, the focusing 
process stops. 

The rules themselves take the form "IF 
Sharpness IS <descriptor1 > AND 
ChangeJn_Sharpness IS<descriptor2> THEN 
Correction SHOULD-BE <value>". The two 
descriptors are labels for fuzzy sets defined 
over the possible values for Sharpness or 
Change_in_Sharpness, respectively. The 
value is a signed number corresponding to 
the magnitude of the focusing adjustment 
suggested. 

In the current system, the range of possible 
Sharpness values is mapped onto six 
overlapping fuzzy sets. These sets roughly 
correspond to linguistic labels such as "very 
poor", "poor", "medium", "moderately sharp", 
"sharp", and "very sharp". Each of these sets is 
simply a function that assigns an integer value 
from 0 to 127 to any Sharpness value. A set of 
membership functions is illustrated in Figure 3. 



Figure 3 
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Similarly, Change__in_Sharpness has six 
membership functions. Thus, the system has a 
total of 36 rules. 

Because the fuzzy values overlap, most of the 
time the rule input values ( Sharpness and 
Change_in_Sharpness) match the "IF" 
conditions for two or more rules at levels 
greater than zero. When this occurs, the fuzzy 
inference technique used in this system 
computes a weight for each rule, which is the 
minimum of the two corresponding 
membership function values. All rules with 
nonzero weights are combined by taking a 
weighted average of the suggested outputs. 
For example, if Rule 1 had a weight of .25 and 
suggested an adjustment of +120 and Rule 2 
had a weight of .75 and suggested an 
adjustment of +40, and all other rules had zero 
weight, the resulting suggested adjustment 
would be +60. This weighted-averaging 
technique assures smooth transition between 
adjacent sets of rules, an advantage compared 
to non-fuzzy rules which change actions 
abruptly as thresholds are crossed. 

Special procedures are implemented to 
prevent undesirable situations. For example, if 
the image is not very sharp and does not 
appear to be improving, the rules may suggest 
a large step continuing in the same direction. 
But if such a step would exceed the limits 
defined in the SET/RESET process, the system 
adjusts the step to stay within limits. If this 
adjusted step still fails to improve, the system 
concludes that it has been looking in the wrong 
direction, returns to its starting point, and 
begins searching in the opposite direction. 

SYSTEM PERFORMANCE 

Operating on an 80386-based personal 
computer with an 80387 math coprocessor, the 
system generally reaches its "best focus" in a 
matter of seconds (corresponding to anywhere 
from 3 to about 25 focusing steps), as long as 
the slide contains sufficiently high contrast to 
generate meaningful values on the sharpness 
index. For very low-contrast slides, the system 
may fail to detect any objects and produce an 
error message during the SET/RESET 
procedure; slides that barely pass the initial test 
may cause the system to "search high and low" 
for any object, or oscillate between two settings 
without converging. It would be a simple matter 
to modify the code to detect this condition and 
terminate with an appropriate error message. 

When the system reaches its "best focus", this 
means it has arrived at what it considers to be a 
sharp image. In some cases, this may differ 
from a human observer’s judgment. Typically, 
it is possible to improve slightly on the 
sharpness of an automatically focused image 


by entering MANUAL mode and making fine 
adjustments. Part of this shortfall can be 
accounted for by the time-versus-quaiity 

tradeoff implicit in the focusing strategy and by 
the limited resolution of the microscope's 
motor; a more sensitive motor controller and a 
greater sensitivity to small changes in the 
sharpness value could increase the value of 
the sharpness function, but generally the 
magnitude of this improvement would be small. 

The other component of the shortfall may be a 
mismatch between the sharpness index as 
computed by the system and the user's own 
concept of what constitutes a sharp image. 
Although preliminary investigations indicate 
that the current sharpness index performs 
adequately, more study is needed to determine 
if significant improvements can be obtained by 
defining the sharpness function differently. 
This may be particularly important at lOOOx 
magnification, where greater shortfalls have 
been observed, possibly due to the more 3- 
dimensional nature of the lOOOx images. 

A side issue, arising during a pilot evaluation 
study with microbiologists as subjects, relates 
to the system's visual display quality. The 
standard- video, black-and-white images 
displayed on an ordinary video monitor were 
judged inadequate for practical used by 
microbiologists. Although full-color, high- 
resolution equipment would have solved this 
problem, it would have added significantly to 
the cost of the prototype demonstration system 
while shedding little additional light on the 
major technical issues addressed in the 
demonstration. 

Even when the "best focus" agrees well with 
human observers' ratings, a second quality 
factor needs to be evaluated: the speed or 
efficiency with which the system reaches its 
focus. As already mentioned, the current 
system performs adequately, but occasionally it 
appears to make adjustments that a human 
observer would consider unusual. 
Improvements might be effected by modifying 
the rule set, fine-tuning the membership 
functions, or selecting an alternate method of 
reconciling overlapping rules. 

To summarize, the current demonstration 
prototype has succeeded in its primary goal, 
which was to demonstrate the feasibility, low 
cost, and relative simplicity of a fuzzy logic 
approach to a complex control problem with 
space system applications. The system can 
focus most slides quickly and with adequate or 
better sharpness, although improvements on 
both factors should be possible. 

TECHNICAL ISSUES 

The purpose of the current prototype system 
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was to demonstrate the technical feasibility of 
fuzzy control for problems like microscope 
focusing. A working production version of such 
a system would require upgraded hardware 
and reprogramming (perhaps using a stand- 
alone microprocessor instead of a general- 
purpose computer). In addition to those 
changes, three technical issues need further 
study. 

Defining the " sharpness " function 

In the final analysis, even the most efficient 
system will fail to be effective if it is optimizing 
the wrong sharpness index. It is critical that the 
peak of the sharpness function correspond 
closely to the location human experts would 
select in manual mode. Beyond that constraint, 
it would be desirable if the sharpness function’s 
shape were conducive to a fast and successful 
search. 

The current method, a sum of squared local 
contrasts, works well for many images, but has 
some drawbacks. When there are two or more 
objects in different focal planes, for example, a 
larger object contains more pixels, and 
therefore contributes more to the sharpness 
index than a smaller one. Also, a "solid- 
colored” object exhibits local contrast only 
along its borders, while a striped, spotted, or 
other textured object shows contrast throughout 
its area and therefore contributes 
disproportionately to the sharpness index. 
Particularly at lOOOx, some images also tend to 
produce optical artifacts (reflections or 
interference patterns) that appear as 
alternating bright and dark bands, generally 
around an object’s edge; these bands are not 
real objects, but by contributing high contrast 
values to the sharpness index, they may throw 
the focusing process off. Finally, high-contrast 
extraneous objects such as dust particles, 
scratches on the slide, air bubbles, or water 
droplets tend to be located on the top and 
bottom of the slide and may "decoy" the 
focusing process away from the true targets. 

The current method is only one of a vast family 
of possible "statistical" indices of sharpness. 
The computation of local contrast might use a 
5x5 or larger region instead of a 3x3, or might 
weight the adjacent pixels differently. Instead 
of sum-of-squares, one could use the sum-of- 
absolute-values, the number or percent 
exceeding a given threshold, or the maximum 
contrast. Other statistical methods might use 
measues based on gray-scale or contrast-level 
entropy, statistical filters, or correlations. 

These "statistical" sharpness measures share 
one drawback: they try to use the same index 
of sharpness for many different kinds of images 
in varying circumstances. By contrast, a human 
microbiologist knows what kind of slide is being 
mounted and has a pretty good idea of what to 


look for. For example, on one type of slide, a 
microbiologist may be interested only in a 
particular band of tissue, and in particular may 
need to know whether the band is a single unit 
or a set of adjacent components arranged end- 
to-end. If the crew member who collected each 
slide could simply indicate which type of slide it 
was, a sophisticated system could use expert 
rules to select the most appropriate statistical 
measure for each one. (A further extreme of 
sophistication might use advanced techniques 
in artificial intelligence, fuzzy logic, or neural 
nets to categorize the slide automatically and 
zero in on features of high interest; this would, 
however, go well beyond the original scope of 
this project.) 


Strategies and tradeoffs for 
” optimization ” 

In defining an "optimal" search strategy, we 
implicitly make trade-off judgments about the 
system's desired performance. Once a 
sharpness measure has been chosen, each 
slide image could potentially generate a 
sharp ness- vers us- st age-position function 
defined over the range of possible stage 
positions. The problem is to arrive "sufficiently 
close" to the maximum value in an "acceptably 
short" time, with "high reliability". Although the 
system designer must work from an educated 
guess, the final evaluation of a system’s 
performance must address all three issues, and 
must agree with the user community’s 
subjective perceptions. 

In the case of microscope focusing, there are 
three conditions that may complicate the 
focusing strategy: 

(1 ) Multiple local maxima. Since it is not known 
at the outset what the maximum achievable 
sharpness index is, there is no guarantee 
that any local maximum is in fact the global 
maximum. Flowever, if the sharpness 
function is reasonably well-behaved (free of 
tall spikes), it should suffice to start by taking 
coarse steps, then progress to finer 
adjustments, 

(2) Plateaus. Sometimes, the sharpness index 
is fair to poor, but small to moderate-sized 
adjustments do not yield significant changes. 
This occurs often when the image is low- 
contrast, Improperly lit, or far out-of-focus. 
The current system's approach is to continue 
in one direction until the sharpness index 
changes or until the upper or lower limit is 
reached; often this strikes the user as 
inefficient "blind searching”. One possible 
approach might be to use a different kind of 
sharpness function until a certain quality of 
focus is reached, then transfer over to the 
"real" sharpness function. Along with this, 


the rules might be modified (or an auxiliary 
set of rules developed) to take larger steps 
when on a plateau. 

(3) ’’High-frequency” fluctuations. Particularly in 
the neighborhood of the maximum 
sharpness, very small adjustments seem to 
produce fairly large jumps in the calculated 
sharpness index. Some of this may be due 
to the resolution of the image digitizer, some 
to variations in the light source, and some to 
uncontrolled motion of the microscope or 
slide. Even at its finest adjustment, the 
stepper motor does not return the stage to its 
exact original position after one step up and 
one step back down. If these variations 
reflect the system's noise level, it would be 
advisable to modify the search strategy (or 
the sharpness function) to keep the system 
from getting caught in a prolonged series of 
very minor adjustments, if they represent the 
sharpness contributions of several values at 
distinct but close focal distances, there may 
be no universally satisfactory way to choose 
among them; the best solution may be to get 
the microbiologist into the decision loop, or 
to transmit several images instead of one. 
As in the case of the plateau, there might 

need to be a different set of rules for the cass 
where sharpness is very good. 

Fuzzy rule Implementation 

The current system uses one of several 
possible interpretations of fuzzy rules. In 
essence, it computes the degree to which the 
input data match the "IF” pattern for each rule 
and uses it as a weight for the numerical value 
specified by the "THEN” portion of the rule, 
taking a weighted average to reconcile when 
multiple conflicting rules "fire". This approach, 
which was developed by Apt Instruments, Inc., 
is conceptually simple and computationally 
efficient, but there is no evidence to date that 
compares its performance with other possible 
techniques. Therefore, it might be both 
instructive and prudent to compare several 
techniques to determine relative performance. 

Another area for improvement would 
incorporate some sort of feedback or adaptive 
learning into what is now a static set of fuzzy 
rules and membership functions. This would 
most likely begin with an intuitive fuzzy model 
based on an expert's behavior or verbal 
instructions and, then, modify the rules and 
membership functions based on experience. 
One possible use might be to begin with a 
generic set of fuzzy rules and then develop a 
specialized version for each different type of 
slide. 

CONCLUSION 

The authors have demonstrated the feasibility 


of using a fuzzy rule-based system to control 
the automatic focusing of a microscope. 
Although several enhancements have been 
suggested, the current demonstration prototype 
illustrates that a very simple rule base and 
inference engine can be used to guide the 
focusing process successfully. The ability to 
produce working prototypes quickly and at low 
cost, coupled with the possibility of capturing 
the control process on small stand-alone 
processors, suggests that fuzzy rule-based 
systems may be an attractive way to implement 
automation in space. 
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